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ABSTRACT 


^  A  frequently  occuring  problem  is  to  find  that  probability  distribu- 
tlon  (PD)  lying  within  a  set  d  which  minimizes  the  I-divergence  between 

K 

it  and  a  given  PD  R.\  This  is  referred  to  as  the  I-projection  of  R 

- -  t 

onto  d.  Csiszar  (1975)  has  shown  that  when  d  *  fl  d  is  a  finite 

1  1 

Intersection  of  closed,  linear  sets,  a  cyclic,  iterative  procedure  which 

projects  onto  the  individual  d^  must  converge  to  the  desired  I- 

projection  on  d  providing  the  sample  space  is  finite. 

*TIUi.e  we.  propose^n  iterative  procedure  which  requires  only  that  the 

be  convex  (and  not  necessarily  linear)  which  under  general  conditions 

t 

will  converge  to  the  desired  I-projection,of  R  onto  fi  d  . 

1  1 
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AN  ITERATIVE  PROCEDURE 


FOR  OBTAINING  I-PROJECTIONS 
ONTO  THE  INTERSECTION  OF  CONVEX  SETS 


1.  Introduction 

Suppose  P  and  Q  are  probability  measures  defined  on  subsets 
of  the  finite  set  Z,  which  ULOG  we  take  to  be  the  first  m  positive  in¬ 
tegers.  The  I-divergence  of  P  with  respect  to  Q,  also  called  the 
Kullback-Leibler  information  number,  cross  entropy  between  P  and 
0,  information  for  discrimination,  entropy  of  P  relative  to  Q,  etc., 
is  given  by 

(1.1)  I(Pl|Q)  -  I  P(k)  in 

k-1 

where  p  *  (p(l) , . . . ,p(m))  and  q  -  (q(l) . , , , .q(m))  denote  the 
probability  mass  functions  (PMF)  of  P  and  0  respectively.  (In 
general,  we  shall  Indicate  a  measure  on  subsets  of  Z  by  an  upper 
case  letter  (P,Q,R,S)  and  the  associated  Radon-Nlkodym  derivative 
with  respect  to  counting  measure  '(PMF)  by  the  corresponding  lower  case 
letter  (p,q,r,s).) 

We  mention  that  I(pUq)  is  defined  analogously  for  general  prob¬ 
ability  measures  on  infinite  spaces,  but  for  simplicity,  we  will  only 
consider  finite  sets.  (See  Kullback  (1959),  or  Csiszar  (1975)  for  the 
general  definition.) 

It  is  well  known  that  I(P[|Q)  *  0,  and  that  I (P |Jq)  *0  iff  P«so. 
Thus  it  is  huerlstlcally  reasonable  to  think  of  I(pI|q)  as  representing 
a  "distance”  between  P  and  Q.  However,  I(*[l*)  is  not  a  metric,  nor 


is  the  symmetrized  version 


J(P||Q)  . 

used  by  Jeffreys  (1948). 

Nevertheless, 

m  l 

(1.2)  I  |p(k)  -  q(k)|  £  [2  I(P[[Q)P  , 
k“l 

as  shown  independently  by  Kullback  (1967),  Kemperman  (1967),  and  Csiszar 
(1967),  so  that  we  have  some  idea  of  what  small  values  of  I(P|Iq)  imply. 

If  we  Interpret  I(p|jQ)  as  distance,  it  seems  natural  to  define 
the  I-projection  of  the  probability  distribution  (PD)  R  onto  a  set 
3  of  probability  distributions  as  being  a  PD  Q  €  3  such  that 
I(Q||R)  <  -  and 

(1.3)  I(Q|[R)  -  min  I(P||R). 

pes 

In  some  sense,  Q  is  the  PD  closest  to  R  that  lies  within  3. 

Minimization  problems  of  the  form  (1.3)  play  a  key  role  in  the 
information-theoretic  approach  to  statistics  (e.g.  Kullback  (1959), 

Good  (1963),  etc.)  and  also  occur  in  other  areas  such  as  the  theory  of 
large  deviations  (Sanov  (1957))  and  maximation  of  entropy  (Rao  (1965) 
and  Jaynes  (1957)). 

However,  in  statistical  circles,  I-projections  are  probably  most 
important  for  being  dual  problems  to  certain  log-linear  model  maximum 
likelihood  estimation  (MLE)  problems.  In  particular,  it  is  known  that 
the  multinomial  MLE  problem 


I 


P<k)StO,Vk  k-1 
|  p(k)-l 
In  p 

(where  U[  is  some  subspace  of  Rm  containing  the  constant  vectors)  has 
precisely  the  same  solution  as  the  I-projection  problem 

(1.5)  Min  KPjJu) 

P :  S-P  €  Mx 

(where  s(k)  *  n(k)  (^  n(i))  \  and  U(k)  ■  ^  is  the  uniform  PMF) .  Note 
that  for  a  subspace  spanned  by  the  vectors  a^ , • • • ,  we  have 

Tit  -  U1,“*.at}A  -n£ai3'L» 

and  hence  (1.4)  is  equivalent  to 

Inf  I 

(s-£ai3x) 

Since  the  S-(a^}X  are  linear  spaces,  Csiszar's  algorithm  of  cyclic,  iter¬ 
ated  I-projections  is  appropriate  here.  In  this  format,  it  is  easy  to 
see  the  connection  between  Csiszar's  procedure  and  the  IPFP  (iterative 
proportional  fitting  procedure)  which  has  received  so  much  attention  in 
the  general  area  of  categorical  data.  Meyer  (1980)  has  an  extensive  dis¬ 
cussion  and  several  examples  where  he  relates  Csiszar's  procedure  and  the 
general  IPFP . 

Suppose  now  that  ♦•••+K  is  a  closed,  convex  cone  expressible  as  a  d 

rest  sum  of  closed,  convex  cones  containing  the  constant  vectors  rather  than  a 


direct  sum  of  subspaces.  Such  a  configuration  would  arise  naturally  if 
one  were  considering  order  constraints  in  a  log-linear  model.  Then  it  is 
well  known  that  the  dual  (polar)  cone  of  J7j,  defined  as 

m 

m  £y;  Jy(i)x(i)  £  0  for  all  x  €  7^3 

is  expressible  as 

tr?  «  0^4*2+* •••*  )*  -  KjriK*n  •  ••okJ. 

In  unpublished  results,  Dykstra  has  generalized  the  (1.4)-(1.5)  duality 
results  to  the  case  where  is  a  closed,  convex  cone  (with  TV?  replacing 
Tit  in  (1.5)).  This  means  that  many  MLE  problems  involving  partial  orders 
in  log-linear  models  are  equivalent  to  I-projection  problems  of  the  form 

Min  I(Pllu) . 

P€&  (S-Kj) 

If  the  K*'s  are  not  subspaces,  Csiszar's  cyclic,  iterated  scheme  need 
not  work.  However,  the  procedure  described  in  this  paper  will  work  since 
the  sets  S-K*  will  be  closed,  convex  sets  of  PD's. 

Of  course  we  would  really  like  to  be  able  to  identify  structure  in 
these  log-linear  model  situations,  which  leads  to  the  area  of  inference 
for  various  competing  models.  While  these  are  important  questions,  we 
shall  only  be  concerned  with  the  MLE  problem  in  this  paper. 

Csiszar  (1975)  discusses  I-projections  in  great  detail,  and  has 
a  "geometric"  development  for  I-projections  which  is  quite  appealing. 
(Cencov  (1972)  also  has  a  geometric  development  of  I-projections, 
but  with  the  arguments  interchanged.)  Cslzar  also  discusses  the  exist- 


ence  of  I  projections,  and  shows  that  if  6  is  a  convex  set  of  PD's 
which  is  variation  closed,  the  unique  existence  of  the  I-projection 
of  R  onto  6  is  guaranteed  provided  there  exists  a  P  £  <S  such  that 
I(P||R)  <  •».  This  result  is  clearly  applicable  for  closed,  convex  sets 
of  PD's  on  the  finite  set  Z.  We  shall  make  repeated  use  of  the 
elegant  characterization  of  l-projectlons  given  in  the  following 
theorem. 

Theorem  1.1  (Csiszar).  A  PD  Q  €  &  (such  that  I(Q||R)  <  •»)  is  the  I 
projection  of  R  onto  the  convex  set  £  of  PD's  iff 

(1.4)  i(pjR)  i  i(p||q)  +  kqIJr)  Vp  £  S. 

Note  that  in  our  setting  of  finite  Z,  it  follows  from  (1.1)  that  (1.4) 
is  equivalent  to 

(1.5)  E  (p(k)  -  q(k) )  in(;4|£)  i  0  VP  £  <*• 

k-1  rW 

If  in  fact  Q  is  an  algebraic  inner  point  of  <J,  i.e.  for  every 
P  £  <f(Pl*Q) ,  there  exists  0  <  a  <  1  and  P'  £  S  such  that 
Q  *  aP  +  (l-a)P',  equality  must  hold  in  (1.4)  and  (1.5). 

This  situation  is  roughly  akin  to  projecting  onto  subspaces  in 
least  squares  theory.  In  particular,  Csiszar  defines  an  £  to  be 
a  linear  set  of  PD's  if  P,  P'  €  6  implies  uP  +  (l-o)P'  £  $  for 
every  a  for  which  it  is  a  PD.  If  6  is  a  linear  set,  then  the 
Inequality  sign  in  (1.4)  and  (1.5)  may  be  replaced  by  an  equality  sign 
as  long  as  Z  is  finite.  Based  upon  this  characerization,  Csiszar 


is  able  to  prove  that  if  X  is  finite,  8  *  fl  <?.  is  a  finite 

1  1 

intersection  of  arbitrary  linear  sets,  and  there  exists  a  P  €  8  such 

that  I(PjjR)  <  *,  then  successive  ,  cyclic,  iterated  I-projections 

onto  the  individual  sets  must  converge  to  the  I-projection  of  R  onto 

<?.  Thus  if  Qq  *  R,  and  denotes  the  I-projection  of  Qn-1 

onto  8  (where  8  m  8.  if  n  -  it  +  i,  1  <  i  <  t) ,  then  Q_ 
n  n  l  —  —  n 

must  converge  to  Q  €  8  as  n-*»  where  I(Q||R)  -  min  I(P||R). 

P €<? 

This  result  is  very  much  dependent  upon  the  assumption  that  the 
8 ^  be  linear  sets  (in  fact  it  is  not  true  in  general)  and  the  accompany 
ing  fact  that  equality  holds  in  (1.4). 


2 .  The  Procedure 

We  now  propose  a  procedure  which  will  enable  one  to  obtain  I- 
projections  onto  a  finite  Intersection  of  arbitrary  closed,  convex 
sets  of  PD's  by  iteratively  finding  I-projections  onto  the  individual 
sets.  We  will  prove  that  that  under  a  mild  restriction,  the  procedure 
must  give  the  correct  solution,  and  then  examine  an  example. 


First  let  us  note  that  we  can  still  define  I-projections  for 
non-negative  vectors  which  are  not  normed  to  be  PMF's  onto  sets  of 


PD’s.  That  is,  for  r  >_  0  (r*0) ,  Q  £  tf  is  an  I-projection  of 
r  onto  a  set  of  PD's  3  iff 


Z  q(k)  in  -  I(QjjR)  -  min  I(P||R)  -  min  Z  p(k)  in£^ 

i  p es  p es  i  rw 


and  I(Q||R)  <  «. 

Of  course,  since  3  contains  only  PD's,  the  I-projection  of 

r  onto  a  convex  set  3  is  identical  to  the  I-projection  of 
m 

r'  -  r/  Z  r(k)  (*  normalized  to  be  a  PMF)  onto  <?.  It  is  easily 
1 

shorn  that  the  characterization  of  I-projections  given  in  (1.4)  and 

(1.5)  is  still  valid,  even  though  I(P|jR)  >_  0  need  no  longer  be 

true.  (We  note  that  multiplication  and  division  of  vectors  will  refer 

to  the  operations  being  performed  coordinatewise . ) 

Let  us  now  state  our  algorithm.  We  assume  we  wish  to  find  the 

t 

I-projection  of  r  onto  3  m  0  3  ,  where  3.  are  closed,  convex 

1  1  1 

sets  of  PD's.  We  assume  that  we  can  project  onto  each  <j^  individ¬ 
ually  and  shall  denote  the  I-projection  of  S  onto  3^  by  ^(dEjjs). 
We  also  assume  there  exits  a  PD  T  €  3  such  that  I(T||r)<  ». 


1.  Let  Sj  j  *  R,  and  let  9^  ^  ■  P(<Sj|Sj  ^).  We  then  set 


81,2  “  Pl,l  "  r  s. 


.  (We  note  that  if  s  (k)  *  0,  then 


so  is  p.  .(k).  We  take  0/0  to  be  1.) 

1  i  P 

2.  Let  ?12  -  *W2IS1,2)#  Set  ®1,3  "  Pl,2  *  r  7^  s^ 

3.  Continue,  until  P.  *  ^(tf  Is.  )  where  “  Pi  ia 


l,t-l 


.  Set  s„ 


.  rPli2  Pl.t 


Note  that  8 


S3 >XSBS33.T; 


T^T^7. 


i 


Pi  1 

2  I  "  Pi 

2 » 1  3 j  j 


4.  P 


2jl-Wl|S2il).  Set  s2>2  -  r 
or  equivalently,  8,  »  -  p,  . /(”*%• 

LtL  Z,I  ^  2 

p 

5.  Continue.  In  general,  set  s  =  p  . *  2  <  i  <  t 

n*i  n‘1_1  'Vl,/  "  “ 

.  ■  p  ,  /(  n~-*-V  We  then  let  P  ,  =  &($.  Is 

n,l  n-l,t  Va^j  j/  n,i  i'  n,: 


and  set  s 


and  define  S  .,,(S  .  if  i  =  t),  etc. 

n,l+l  n+i,l 


Suppose  now  that  the  are  actually  linear  sets  so  that  equality 

holds  in  (1.5).  Noting  that  for  any  P  € 


(2.1)  I(PUsn  ±)  ■  £  POO  Ai[p(k)/sn  |k)] 

-  L  p(k)  -**»[p(k)/(Pn  1_1(k)/pn_ljl(k)/Sn_l  i(k)))] 
k  * 

(2.2)  -  £  p(k)  in  ■  P-Mrr  +  £  p(k)  in  - - , 

k  Pn,i-1 k  8n-l,iW 

* 

we  observe  that  the  last  tern  must  be  equal  to 


£  p. 


n-l.i 


(k)  in 


n-i  ,x 


and  hence  free  of  p.  Thus  the  p  €  which  minimzes  (2.1), 
is  also  the  one  which  minimizes  the  first  part  of  (2.2),  i.e.,  the 
I -projection  of  P  .  onto  &  .  It  easily  follows  that  our  procedure 

"*  tl  1 1"  A  A. 

reduces  to  the  cyclic,  iterative  procedure  given  by  Csiszar  when  the 


are  closed  linear  sets. 


For  a  simple  example  to  show  that  Csiszar's  procedure  does  not  work 


for  general  convex  sets,  consider  the  following: 


f/Pll’  P12\ 
\\p21»  P22 /’ 


P11  *  P12*  P21  *  P22*  pij  * 


P11  *  P21*  P12  *  P22*  Pij  * 


22  1 

0,  EtPy  *  lj, 

22  } 

o,  EE  P±J  -  i| , 


and 


Csiszar's  procedure  yields 


whereas 


solution. 

Everything  hinges  on  the  following  theorem. 


is  the  correct 


Theorem  2.1.  Assume  df  -  H  d^,  where  the  d^  are  closed,  convex  sets  of 


PD’s  and  R  j4  0  is  a  nonnegative  vector  such  that  there  exists  a  T  € 


where  I(t||r)  <  ®.  If  there  exists  a  convergent  subsequence  p  .  -*■  p 

Dj  ,  l 

for  some  i  such  that 


P_ 
n.  y  i 


(2.3)  lim  inf  Z  (p.  ,  (k)  -  p(k))  ln(— /Yv)  > 

i  k  "j*1  Sn,,iW 


for  every  i,  then  pR  ^  -*■  p  as  n  -*■  »  and  P  *  ^(<J|R)  where 


p  and  s  are  defined  as  in  the  proposed  algorithm. 

Hyi  Ilf  1 


Proof.  Recall  that  p  .  -  . \s  )  where 


(2.4) 


^n,l  ^n,i-l  ^n-l,t 


»  .  *-  •  •  • 

H| i  S  <  S  .  .  S  « 

n»l  n,i-l  n-1,1+1 


wfe-i) 


if  i  =  1 


if  2  <  i  <  t. 


e  Vi  “(r*7 >  -Ew“(^rt 

n,i  n-l,i 


(2-5)  ■  E  pn.i  +  E  P„,i  2  P"<^> 

n»i-i  n-l,i  n-l,i 

-  e  p»,i  +  2<pa,i  -  w> 

n>i  a  n~l*i 

—  ^^n.i^n.i-l^  2  <  i  <  t,  and  similarly  if  i  =  1, 


since  (by  1.4)  the  last  tern  must  be  nonnegative  because  P  .  c  $  . 

n,  l  1 

Noting  that  I(P  .IIP  .  «)  >  0  since  P  .  and  P  .  ,  are  PD's, 
tigi11  nti-l  —  n»l  n,i-l 

we  have  that  I(P^  is  nondecreasing  in  n  for  each  i.  Let 

us  now  show  that  these  sequences  are  bounded  above. 


For  V  €  fl  . 
1 


I(V||P  >  -  E  Vln(^-) 

n,i 


Z  Vin  V  -  £  v*n  r[~£*i  . , 

8n,l 


^n,i  ^n-1 ,i+l 
8n,i  8n-l,i+l 


t  p  . 

(2.6)  -  Z  vln  v  -  Z  vln  r  -  Z  E  vin  -  I(Pa  .||S  )] 

j-1  8a,j  a°  a,J 

t  fn,  j  <  i 

-  E  I(P  Js  <>  where  a  "  | 

j“l  8,3  3,3  |n-l,  j  >  i. 

*I(V||R)  -  E  W.jll.j) 


by  (1.4)  and  the  fact  that  V  belongs  to  every  <J^.  Thus,  choosing 
V  such  that  I ( V|| R)  <  00 ,  we  have  a  uniform  upper  bound  on  I(P  ||S  ) 


12 


Hence  the  lim  I(P  . Us  .)  exists  finite  for  every  i,  and  by  (2.5) 
n  ,  i  n,l 


n-*» 


(2.7)  I(P  . HP  .  ,)  0  as  n  ®  for  2  <  i  <  t 

n,i"  n,i-l  — 


-*■  p  for  some  i. 


(Similarly  I(Pn  1[jPn_1  fc)  -*•  0  as  n  -►  ».) 

Note  that  (1.2)  and  (2.7)  imply  that  if  p  .  . 

t  j*1 

then  p  .  -*•  p  for  every  i.  Thus  p  €  fl  S.  since  p  €  S .  an< 

nj’i  1  1  V1  1 

the  <S.  are  closed. 

1  t 

Using  (1.5)  and  (2.3),  for  any  V  €  fl 

1 

Pn.  ,i 

0  <  E  lim  inf  E(p-  ^  “  pHn( — ^ — ) 
i  j  k  V1  8nj,i 

Pn.,i 

+  EE(v  -  p  ,)£n(-“^--) 

Ik  “j’1  8n  ,i 

*  K  4 

n  ,i 

<  lim  inf  E  E  (v-p)in(-  ) 

j  i  k  nj,i 

PV* 

■  lim  inf  E  (v-p)  r  II  ( — ^  — )  /r 

i  k  i  V,.‘ 

V.t 

-  lim  E  (v-p)  tn( — £-— ) 
i  k 

■  E  (v-p)  *n(£). 

k  r 


Thus  P  ■  |R)  by  (1.5).  Setting  V  -  P  in  (2.6)  and  using  (2.3), 


it 


follows  that  for  the  subsequence  {n^ } , 


13 


£  I(Pn  iHSn  i>^I(PHR)* 

1-1  V1  v 

However,  by  the  nonotonlcity  in  n  of  I(P  11 S  .),  we  have  that 

till*’  "jl 

j,  *  i(p«m- 


Thus  I(P||P  „)  -*■  0,  so  that  by  (1.2)  and  (2.5),  p  .  p  as  n  • 

"  n ,  t  n ,  x 

for  all  1. 

We  remark  that  It  would  take  rather  surprising  behavior  of  the 

p  for  condition  (2.3)  to  not  hold,  and  we  strongly  conjecture  that 
n,i 

this  condition  is  always  true.  As  we  note  in  the  following  corollary, 

if  the  p  .-jfcP(<*|R),  we  must  have  sup  51  8  .  (k)  »,  and  we  have 

n,t  n,i  k  n»i 

been  unable  to  construct  examples  where  this  happens.  We  point  out 

that  when  one  uses  the  algorithm,  they  can  put  in  a  step  to  check  the 

value  of  sup  51  8  . (k) .  If  the  algorithm  is  not  going  to  converge 
n,i  k  n,i 

correctly,  then  this  value  must  become  excessively  large.  Otherwise, 
the  algorithm  must  converge  to  the  correct  solution. 


Corollary  2.1.  If  the  algorithm  should  not  converge  correctly,  then 

sup  51  8  .  00  -*■  ®  88  n  «. 

n,i  k  n’i 


Proof.  Suppose  s  . (k)  is  uniformly  bounded  above,  and  p  .  -*•  p 
n ,  1  n  ^ ,  l 

is  a  convergent  subsequence  (which  must  exist  since  0  <  p  . (k)  <  1). 

—  n,i  — 

If  there  exists  0  <  m  <  s  . (k)  for  all  n  ,k,  then  (2.3)  follows 


by  standard  continuity  properties.  If  not,  there  exists  at  least  one  k 
such  that  s  . (k)  takes  on  arbitrarily  small  values.  Then  either 

Y1 


14 


p  .  (k)  -*■  0  for  all  such  k  (which  cause  no  problems) ,  or  there  exists 

tij.i 

a  k  such  that  p  .(k)  p(k)  >  0,  while  s  .  (k)  takes  on  arb- 

V1  V1 

trarlly  small  values.  This  contradicts  the  uniform  upper  bound  of 


We  recommend  that  when  one  uses  the  algorithm,  they  shenld  compute  t.he 

,  t 

average  value  over  an  entire  cycle  (•£■  E  p  ,)  rather  than  a  single 

c  1-1  0,1 

projection  p  to  estimate  the  I-projectlon.  Convergence  Is  still 
n,i 

guaranteed,  and  this  value  Is  much  more  stable  and  seems  to  converge  much 
more  quickly  to  the  correct  solution. 


3 .  An  Example 

To  Illustrate  our  algorithm,  we  consider  n*n  arrays  of  probabilities, 

n  n 

say  (p.  ),  Pt.  ^  0,  k, j  -  l,...,n,  E  E  p.  -  1.  We  denote  the 
3  J  j-lk-1  Rj 


corresponding  marginal 
1 , . . . ,n. 


n 

PD’s  by  p.#-  E  p. 

*  j-1  *3 


n 

and  p  -  E  p.  ,  k,j  - 
3  k-1 


We  now  consider  the  problem  of  finding  the  I-projectlon  of  a 
fixed  array  (r^j)  subject  to  the  marginal  PD's  being  stochastically 
ordered,  i.e. 

1  1 

i  Pf,  1  Ep  ,  for  all  1. 

1-1  1-1 


(Kullback  (1971)  has  given  an  iterative  procedure  for  I-projections  where 
equality  is  forced  to  hold  for  all  1,  also  known  as  marginal  homogeneity. 
Equivalently,  we  want  to  find  the  I-projection  of  (r  )  onto 

kj 

$-  fl  where  $  “((p..);  E  E  p,  *  E  E  p,  }. 

11  1  k3  m-ll-1 


(3.1) 


Note  that  the  are  closed,  convex  sets  of  PD' s  which  are  not 

linear  sets.  I-projections  of  (v.  ,)  onto  the  d.  can  be  found  by  fore 

J  ^ 

ing  equality  in  the  constraint  if  the  (v,  ,)  violate  the  constraint.  (S 

J 

Theorem  2.11  of  Barlow  et  al.  (1972)  which  can  be  modified  to  apply  to 
arbitrary,  convex  functions.) 

To  express  P(^|v),  we  let 

•  {(t,m) ;  I  _<  l  _<  i,  i+1  S  ®  _5  n}, 

•  {(l,m);  1  <  m  £i,  i+1  <1  ^n), 

C±  *  {(£,m) ;  £,m  -  l,...,i)  U  {(i,m);  i,m  «  i+1, . . . ,n} 


and  v  «  (E  v  E  v  )  . 

1  a  d  *,m 

Ai  Di 


If  <vv<>  satisfies  the  constraint  of  &  (E  v„  *  E  v#  ),  then 

1  A.  *’m  B. 


P(<*Jv)  ,  “  v  J  v,  for  all  k,j. 

1  k,j  k,j  j  j 

If  (v..)  does  not  satisfy  the  constraint  of  d  (E  v  <  E  v  )  then 

i  a  *  pin  «  £*in 

Ai  i 

(\  vt.»\  H 

'kApzJ  [2nv-rl-<M)£v 


k,J  V  E  V 


*,« j 


i2  V  +  £  V  J’1,  (k,j)  6  B 

i  r  &,m  i 


[2  v  +  E  v#  l"1  ,  (k,j)  fc  C,  . 


The  key  point  is  that  finding  the  I-projection  onto  is 

quite  easy  (and  easily  programmed),  while  finding  the  I-projection  onto 
n-1 

&  *  fi  is  very  dlfllcult.  However  our  algorithm  enables  one  to  find 
the  latter  I-projection  using  only  the  ability  to  handle  the  I-projections 
onto  the  individual  d^. 

To  illustrate  our  example  with  some  numbers,  we  consider  some  rather 
famous  data  from  Stuart  (1953)  concerning  grades  of  unaided  distance  vision 
for  left  and  right  eyes.  If  one  wished  to  estimate  the  probabilities  of 
falling  into  the  various  categories,  subject  to  the  provision  that  right 
eye  vision  is  at  least  as  good  as  left  eye  vision,  one  might  find  the 
I-projection  of  the  data  in  Table  1  onto  the  8  given  in  (3.1).  Using 
this  algorithm,  we  have  essentially  obtained  convergence  to  the  true 
I-projection  by  3  cycles.  These  values  are  listed  in  Table  2  (with  the 
unrestricted  MLE's  given  in  parentheses). 

This  estimate  might  prove  useful  for  constructing  a  likelihood  ratio 
type  test  for  testing  whether  right  eye  vision  is  better  than  left  eye 
vision.  This  data  is  treated  by  Plackett  (1981)  who  uses  it  for  tests 
Involving  marginal  homogeneity  and  quasi-symmetry.  It  also  appears  in 
Kendall  (1974). 

4 .  Acknowledgement 

The  author  would  like  to  thank  a  referee  and  the  associate  editor  for 
helpful,  knowledgeable  comments  and  Peter  Wollan  for  doing  the  calculations 


in  Table  2. 


RIGHT  EYE 


Table  1 

Unaided  Distance  Vision 


(From  Kendall  (1974)) 


LEFT  EYE 


Highest  grade 


Second  grade 


Third  grade 


Lowest  grade 


Totals 


Highest 

grade 

Second 

grade 

Third 

grade 

Lowest 

grade 

821 

112 

85 

35 

116 

494 

145 

27 

72 

151 

583 

87 

43 

34 

106 

331 

1052 

791 

919 

480 

Totals 


Table  2 

I-Prolection  of  Data  in  Table  1 


(Values  in  parentheses  are  Table  1  values  normed 
to  sun  to  unity.) 


Highest 

Grade 


„  .  .2534 

Highest  grade  (  2532 


Second  grade 


Third  grade 


,  *  .  *0120 
Lowest  grade  (<0133) 


LEFT  EYE 


Second 

Grade 


.0095 

(.0105) 


Totals 


.3234 

(.3245) 


Third 

Grade 


Lowest 

Grade 


.0120 

(.0108) 


.1022 

(.1021) 


.1532 

(.1480) 


Totals 


.3260 

(.3247) 


.1532 

(.1586) 


.9999 

(.9999) 
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